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Abstract: The question whether a time series behaves as a random walk or as a stationary 
process is an important and dehcate problem, particularly arising in financial statistics, 
econometrics, and engineering. This paper studies the problem to detect sequentially that 
the error terms in a polynomial regression model no longer behave as a random walk but as 
a stationary process. We provide the asymptotic distribution theory for a monitoring pro- 
cedure given by a control chart, i.e., a stopping time, which is related to a well known unit 
root test statistic calculated from sequentially updated residuals. We provide a functional 
central limit theorem for the corresponding stochastic process which implies a central limit 
theorem for the control chart. The finite sample properties are investigated by a simulation 
study. 
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1 INTRODUCTION 

Random walks have been proposed as reasonable models for discretely observed data in 
many disciplines. In engineering, they have been proposed to model production processes 
with degradation. For instance, the additive damage model assumes that damage cumu- 
lates yielding a random walk, and the system fails if the cumulative damage reaches a 
threshold. We refer to Birnbaum and Saunders (1969), Taguchi (1981, 1985), Taguchi et 
al. (1989), Adams and Woodall (1989), Doksum and Hoyland (1992), Vander Wiel (1996), 
Durham and Padgett (1997), Park and Padgett (2006), and Srivastava and Wu (1994, 
2003). In financial statistics, random walks appear as a model for the (log) prices of an 
exchange-traded asset. That idea dates back to Bachelier (1900), and nowadays there is 
an extensive literature on the random walk hypothesis in the empirical finance literature, 
mainly addressing the question whether the increments are correlated. Random walks have 
also been proposed as a model for important economic series as the gross domestic product. 
Therefore, an important problem is to check sequentially whether a time scries is compat- 
ible with the random walk model or follows an alternative (out-of-control) model under 
which the series is stationary. 

As is well known, a false answer to that question can lead to completely wrong 
statistical conclusions, since even elementary statistics change their convergence rates and 
limit distributions. The implications for a rich class of nonpar ametric kernel control charts 
covering, e.g., an approximation to the classic EWMA control chart have been discussed 
in detail in Steland (2004). Another popular approach to monitor both i.i.d. observations 
and random walks resp. Brownian motions to detect changes in the mean is based on the 
CUSUM procedure, which is known to be optimal in the sense of Lorden's criterion. We 
refer to Beibel (1996), Moustakides (1986, 2004, 2007), Ritov (1990), Siegmund (1985), 
Shiryaev (1996), and to the monograph of Brodsky and Darkhovsky (2000). Having this in 
mind, it is of particular interest to study sequential monitoring (surveillance) procedures, 
which are designed to detect departures from the random walk hypothesis as soon as 
possible. 

In this article we investigate a sequential monitoring procedure which is related to 
a well known unit root test studied in detail by Breitung (2002). To test the unit root 
null hypothesis against the alternative of stationarity, he proposed to use a variance ratio 
statistic comparing the dispersion of partial sums with the dispersion of the observations. 
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That test statistic is similar to the statistic underlying the so-called variance ratio or KPSS 
test proposed by Kwiatkowski et al. (1992) to test the inverse testing problem of stationarity 
against the unit root alternative. Lee and Schmidt (1996) have shown that the KPSS 
test is also consistent against stationary long-memory alternatives, for a further detailed 
study we refer to Giraitis et al. (2003). The KPSS test is known to be powerful for many 
important data generating processes and robust in terms of the type I error rate. For both 
testing problems (random walk versus stationarity and vice versa) sequential monitoring 
(surveillance) procedures based on control charts related to the variance ratio statistic 
have been proposed in Steland (2007a). In that paper the original time series Yi, 1^2, ... is 
monitored. Under mild conditions the asymptotic distributions of the associated stopping 
times have been established under various in-control and change-point models. 

Motivated by promising results from a prehminary study (Steland, 2006), this article 
considers the more involved and delicate problem to test sequentially whether or not the 
error terms in a polynomial regression model form a random walk, thus allowing for non- 
linear time trends. Assume we observe sequentially a time series {Yt : t e N} of real- valued 
observations satisfying 

Yt = mt + et, t e N, 

with E{et) — for all t. In many applications the regression function m is smooth, which 
motivates to consider polynomials of known degree. Thus, we assume 

Yt^Po + Pit + --- + Ppt'' + €t, teN, (1.1) 

where /3 = (^o, ■ ■ ■ , Pp)' £ MP'^^ are unknown regression coefficients and p e Nq. Basically, 
the aim is to detect a departure from the in-control model that the error terms form a 
random walk in favor of a stationary process. Note that the model covers the case that 
before the change a Brownian motion with polynomial drift, ^{t) — fj,{t) + (TB{t), where B 
denotes standard Brownian motion, cr > is a constant, and ii{t) — Yl^=o ^3^^ i discretely 
sampled at time instances t — 1,2, . . . ,q — l.ln this case = aB{t) ~ A^(0, aH), i.e., the 
variance is a linear function of time. After the change we observe ^{t) — ijL{t) + aB{q) +r]{t), 
t — QtQ + ^t ■ ■ ■> where rjlt) is a stationary process; e.g. given by a continuous-time moving 
average, 

r){t)^ [ ip{t-s)dB{s), 
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for some function with J (p'^{t) dt < oo. Our results allow for substantially more general 
error sequences. 

Since for many practical applications the most important alternative model is a (poly- 
nomial) time trend with stationary errors, we will apply a control chart (stopping time) 
providing a signal, if there is evidence that the errors are no longer compatible with the ran- 
dom walk hypothesis. We provide the asymptotic distribution theory under the in-control 
model that the error terms behave as a random walk but allow for an unknown polynomial 
time trend. Further, we establish results under a change-point model where the errors form 
a stationary process after an unknown change-point. Since our results provide the asymp- 
totic distribution of the stopping time, one may design a surveillance procedure according 
to various criteria. Particularly, our results allow to design the procedure to guarantee a 
specified asymptotic significance level (type I error rate). If we get a signal, the classic poly- 
nomial regression model with stationary errors can be regarded as statistically confirmed, 
which is an attractive property for many applications. 

We study the intuitive approach to calculate the least squares residuals and to apply 
an appropriate monitoring procedure to these residuals. In sequential analysis, recursive 
residuals are often used, see the classic paper by Brown et al. (1975), and Sen (1982), 
mainly because they are fast to compute. However, having in mind contemporary com- 
puting facilities, we introduce sequentially updated residuals, where at each step the full 
set of residuals is calculated. We consider a monitoring procedure with a time horizon T 
where monitoring stops, because in many real applications it is unrealistic to assume that 
monitoring can be conducted forever. However, the modifications of the results to allow 
for infinite monitoring are straightforward and briefly discussed. 

The rest of the paper is organized as follows. In Section |2] we specify and discuss the 
assumptions on the error terms and introduce the proposed procedure and required regu- 
larity conditions. The asymptotic results for the process of sequentially updated residuals, 
the process associated to the proposed control statistic, and for the resulting stopping time, 
are discussed in detail in Section [3] under the in-control model that the regression errors 
behave as a random walk. The results are constructive in the sense that explicit represen- 
tations of the asymptotic error process can be obtained in terms of the moment functions, 
j^t^B{t) dt, s G [0, 1], A; G N, associated to a standard Brownian motion B, which makes 
simulation from the limiting processes feasible. Section H] gives asymptotic results under a 
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change-point model where the behavior changes after a certain fraction of the data from a 
random walk behavior to a stationary process. We report in Section [5] about a simulation 
study which examines some finite sample properties of the method. Proofs of the main 
results of this paper are postponed to appendices. 

2 MODEL, ASSUMPTIONS, AND THE METHOD 

2.1 Model and Assumptions 

It remains to specify model f 1 1.1 1) in detail. We assume that the error terms, = 1^ — m^, in 
model (11. ip form an AR(1) model with possibly correlated but weak dependent innovations 
(for precise assumptions see below), i.e., 

et = ptet-i+uu tGN, (2.1) 

where pt G ( — 1, 1] are unknown parameters. If = p = 1 for all t, {Yt} is a random walk 
and integrated of order 1, /(I). Here and throughout the paper we simply write {Yn} if the 
index set is clear. For |p| < 1 stationary solutions of the above equation exist. 

We consider the following change-point testing problem. The null hypothesis, 

Hq : Pt = 1 for all t, 

states that the error terms of the regression model form a random walk, i.e., are integrated 
of order 1. The alternative Hi = Uq>iH['^^ with 

h['^^ : pt = l, t <q, pt= p, t>q, \p\ < I 

specifies that there exists a change-point such that the subseries {cf : t > q} satisfies 
stationary AR(1) equations. It is important to note that the method proposed in this 
paper does not require any specification of an alternative. In Section H] we introduce a 
specific change-point model related to this testing problem. 

Let us consider an example. 

Example 2.1. Assume et = p{L)^t with i-i-d. A^(0,(t|) for some cr^ > 0, p{z) = 
Yl'j=o^j^^ with coefficients Uj G M, L the lag operator given by Let = Q-i- Suppose that 
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the characteristic polynomial, 1 — p{z), has exactly one unit root of multiplicity 1. Then 
p*{z) = p{z)/ (l — z) can be inverted, and we obtain the representation {1—L)et = p*{L)~^Ut, 
i.e., 

et = ei_i + y^/jjMt-j, 

for certain coefficients Pj, see Brockwell and Davis (1991, Sec. 3.3). Thus, MA(q)-models 
with an unit root appear as a special case for the error terms in model ( li.ij) under the null 
hypothesis. 

Concerning the error terms {ut} we shall assume the following mild nonparametric 
regularity condition making precise our understanding of weak dependence. 

(E) {ut : t G N} is strictly stationary with mean zero and -Eluip < oo such that 

oo 

|Cov(ni,Ui+i)| < oo, 

t=i 

and satisfies a functional central limit theorem (FCLT), i.e., 

J2 Ui^r]B{s), T^oo, (2.2) 

i<lTs\ 

for some constant < ?7 < oo. Here B denotes a Brownian motion with -B(O) = 0, 
and ^ stands for weak convergence in the Skorohod space D[0, 1]. Skorohod spaces 
are briefly discussed at the end of this section. 

Remark 2.1. (i) By the Skorohod- Wichura- Dudley representation theorem (Pollard (1984), 
Ch. IV. 3, Theorem 13), a condition as \2.2) is equivalent to the condition: There are 
Brownian motions B^, T > 1, such that 

sup |T^^/^ Mj — ?7i?T(s) I = op(l), T — )■ oo. 

i<lTs} 

(a) Combining model 112. 1\} with pt = 1 for all t under the assumption (E) yields a 
nonparametric approach to define the I{l)-property of a time series. 

As an example satisfying the assumption (E) let us discuss briefly ARCH{oo) models, 
a popular parametric class of time series models. 
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Example 2.2. Recall that {Xt} satisfies ARCH(oo) equations, if there exists a sequence 
of i.i.d. non-negative random variables {ij}, such that 

oo 

Xj = vS: Vt = a + ^ bjXt_j, 

3=1 

where a > 0,bj > for j G N. Suppose now that 

where {e^} are i.i.d. with E{et) — and E{e^) — 1. Put at — rjt and — ^1 embed 
the classic ARCH model into the above ARCH{oo) framework. Giraitis, Kokoszka and 
Leipus (2001, Example 2.2 and Theorem 2.1) have shown that {ut\ satisfies (E) provided 
E\ei\^ < oo and 

OD 

{Eitf"Y.^,<l. 



2.2 Monitoring Procedure 

Our stopping time defining the detector essentially relies on a weighted version of the KPSS 
test statistic, see Kwiatkowski et al. (1992), Breitung (2002), and Steland (2007a). At each 
time point t <T when a new observation is available, we calculate the full set of residuals 
ei(i), . . . , e~t(t) using all available observations Yi, . . . , Y^. Using these sequentially updated 
residuals, we calculate an appropriately weighted version of the unit root test statistic. 
Define 

= F^EUfW • 

In these formulas called kernel, is a nonnegative function with J K{z) dz < oo. Kernels 
such that K{z) is decreasing for increasing \z\ as the Gaussian kernel or the Epanechnikov 
kernel given by z i— )■ (3/4)l[_i i](l — z"^), z & M., have the intuitive appeal that recent 
summands get higher weights than past ones. However, our main results work under the 
following weak conditions: 

(Kl) \\K\\oo < oo, / K{z) dz = 1, and / zK{z) dz = 0. 
(K2) K is Lipschitz continuous. 
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The parameter h = hx is used as a scaling constant in the kernel and defines the memory 
of the procedure. For instance, if K{z) > for 2; G [—1,1], and K{z) = otherwise, Ut 
looks back h observations. We will assume that 

lim T/hr = C e [l,oo). 

T->-oo 

That condition ensures that the number of observations used by the procedure gets larger 
as T increases. 

The KPSS or variance ratio control chart is defined as 

Rt = mi{k <t<T ■.Ut< cr}, T > k, 

with the convention inf = 00. T is the time horizon where monitoring stops. For our 
asymptotic results we assume T — )■ 00, since for applications approximations of the distri- 
bution of Rt for moderate and large time horizons T are of interest, cr is a control limit 
(critical value) chosen by the statistician. 

It remains to discuss how to choose the control limit cr. Since monitoring stops latest 
at time T, we may interpret the stopping time as a hypothesis test with early stopping 
in favor of the alternative. Thus, one may choose cr to control asymptotically the type I 
error rate of a false decision in favor of stationarity, i.e., 

lim Po{Rt <T) = a, (2.3) 

T— i-oo 

for some given a G (0, 1). Here Pq indicates that the probability is calculated under the null 
hypothesis. Alternatively, one may control a conditional version of the in-control average 
run length (CARL). Note that the stopping time Rt takes values in the set {k, . . . ,T} U 
{00}, where 00 represents no signal, which is the preferred event under the in-control 
model. Now we may choose cr such that CARLq = Eq{Rt\Rt < 00) is greater or equal to 
some given value ^ G {k,T). Since our results provide the asymptotic distribution of the 
stopping time Rt, one may also choose the control limit to control other characteristics, 
e.g., the (conditional) median average run length. For simplicity of exposition we shall 
assume in the sequel that cr is chosen such that (12. 3p holds. 

We will assume that monitoring starts after a certain fraction of the data, i.e.. 



k = [Tk\, for some k G (0, 1), 



Sequential Residuals and Detection of Stationarity 



9 



to avoid that inference is based on too few observations at the beginning. The event Rt < T 
is interpreted as evidence for stationary innovations, and we get that information after Rt 
observations instead of waiting until time T. If Rt = oo, the random walk hypothesis for 
the error terms is regarded as compatible with the observed data. 

2.3 Extension to Infinite Time Horizon 

Suppose we observe sequentially an infinite sequence Yi,Y2, . . . and want to monitor this 
series with the detection rule 

inf{A; < t < oo : Ut < Cr}. 

In this context, T is simply used to define an appropriate time scale to determine the 
bandwidth sequence Ht with T/Ht — )■ C- The FCLT f l2.2p is now replaced by 

J2 Mi : s G [0, oo)} 4 {7]B{s) : s G [0, cx))}, 

i< [TsJ 

as T — )■ oo, where convergence takes place in the space D[0, oo) instead of D[0, 1]. All limit 
theorems in this paper are formulated for the time interval [n, 1], i.e., in the space D[k,, 1], 
but are valid for D[k,, z] for any fixed 1 < z < oo. X„ X in D[0, oo) is equivalent to 

[0,m] 

in D[0,m] for each integer m, where gm(t) = l[o,m-i](^) + {rri — t)l[m-i^rn\(t) , t G [0,oo), 
see Billingsley (1999, Sec. 16) or Pollard (1984, Ch. VI.) Thus, the results carry over to 
D[k,oo), and there is no loss in generality to consider the time interval [0, k]. 

2.4 Skorohod Spaces 

In this paper we will also use the notion of weak convergence in the Skorohod space 
^([0, 1]2;M*--). Denote the Skorohod space of cadlag functions [0,1] ^ M by D[0,1] = 
D{[0, 1]; M). Compared to D[0, 1] the space -D([0, 1]^; M'^) has been only rarely used in the 
literature. Therefore, we close this section with a brief exposition of the most important 
definitions and facts. 
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Recall that a sequence {X, Xn} of random elements with values in a metric space 
converges weakly, denoted by Xn X, as n ^ oo, if Eh{Xn) — >■ Eh{X), n ^ oo, for 
all measureable real functions which are bounded and continuous w.r.t. the metric. For a 
detailed classic treatment of these issues we refer to BiUingsley (1999). 

Equip -D[0, 1] with the Skorohod metric d yielding a complete and separable metric 
space. For p G M let D]rp[k, 1] = D{[n^l\;W) denote the space of all cadlag functions 
[k, 1] W which we equip with the metric dp{f,g) = YILi^ifi, 9i), f = (/i, • • • , /p)', 
g — {gi,...,gpy, fi,gi : [0, 1] — )■ M, i = 1, . . . ,p. The treatment of its generalization 
to the index set [0, 1]^, i.e., D{[0, is more subtle. Let us briefly recall some facts 
about this function space and weak convergence of sequences of -D([0, 1]^; R)-valued random 
elements, as studied by Straf (1970), Bickel and Wichura (1971), and Neuhaus (1971). The 
space D([0, 1]^; R) can be defined as the uniform closure of the vector subspace of all simple 
functions, i.e., linear combinations of functions of the form 1 1-> lEixEQit) where each is 
either a left-closed, right-open subinterval of [0, 1], or the singleton {1}. Here the closure is 
taken in the space of all bounded functions [0, 1]^ — > R. For functions f,gE -D([0, 1]^; R) an 
appropriate metric, d2{f, g), is defined as the smallest £ > such that there exist continuous 
bijections Ai, A2 : [0, 1] — )■ [0, 1] with ||A — id ||oo < £ and ||/ — 5'° A||oo < e. Here A = (Ai, A2) 
and go\{r, s) = g{Xi{r), A2(s)) for (r, s) e [0, 1]^. A sequence {/„} C -D([0, 1]^; R) converges 
to some / G -D([0, 1]^; R) iff there exists some sequence {A„} of pairs of continuous bijections 
[0, 1] — )■ [0, 1] such that o A^ — /||oo and ||A — id ||oo — >■ 0, as n — >■ 00. Further, if 
/ e C([0,1]^;R) convergence in the Skorohod metric implies uniform convergence, since 
in this case / is uniformly continuous. It turns out that (Z^([0, 1]^; R), ^2) is a separable 
metric space, a common framework to define weak convergence of probability measures 
and random elements. 

3 ASYMPTOTIC RESULTS FOR INTEGRATED 
PROCESSES 

This section is devoted to a detailed study of the proposed procedure unter the null hypoth- 
esis that the error terms of the regression model behave as a random walk. Our approach 
is to represent the KPSS control chart as an inf-functional of the stochastic process asso- 
ciated to the sequence {Ut}. That process turns out to be a functional of the stochastic 
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process associated to the residuals up to negligible terms. We provide functional central 
limit theorems for these processes and a central limit theorem for the stopping time Rt- 

We need some notations. Let X„ denote the design matrix for a polynomial regression 
of order p with intercept based on n observations, i.e., 



1 1 

1 2 

1 n 



1 



[xi, . . . , x„] , 



where 

Define for p + 1 < i < T the random vectors 

et = (ei,...,et)' 

et = (I,-X,(X;X,)-ix;)Y„ 

with Yt = {Yi, . . . jYt)'. It denotes the i-dimensional identity matrix. 



3.1 Residual Process without Updating 

Let us first consider the natural process associated to the sequence ei, . . . , of residuals, 
where at time t the current residual is simply added to the residuals e^, [T/tJ < i < t. 
Here the former residuals are not updated. In the sequel [TsJ stands for the current time 
point. The stochastic process associated to ei, . . . , is given by 

where ?t = for < t < p + 1, and 

is the last coordinate of the vector e^Tsj = (ci, • • • , e^[rsj)'- 
We have to introduce the weighting matrix 

Wi = diag(i,r\...,r^), ten, 

to take into account the order of the polynomial regressors. 



12 



Steland 



Lemma 3.1. Fix k G (0, 1). Assume (E). Then 

T-^'/'WLTsjX'L^.jeLTsj f {l,r / s, . . . ,{r / sf)' B{r) dr, m D^,[kM 

Jo 

as T —> oo, where the limit is almost surely (a.s.) continuous. 

Lemma ISTTl plays a crucial role in the proofs of the main results, but it is also interest- 
ing in its own right. Notice that Xj^^jei^r^j is the natural sufficient statistic when the errors 
are i.i.d. normal. The lemma states that for random walk error terms with weak depen- 
dent increments the correct scaling operator for the natural sufficient statistic is given by 
T^'^/^Wlt'.j to obtain a non-degenerate distributional limit. The limit process is given by 
the vector of weighted integrals of Brownian motion, rj f^{r/s)'^B{r) dr, k = 0, . . . ,p, where 
the integral is a Riemann integral. The factor r] summarizes the impact of the correlation 
of the increments. 

Let us introduce the Hilbert matrix of dimension p + 1 given by 

H = (l/(i+ j - l))^je{i,...,p+i}- 
It is known that its inverse, H~^, has entries 

see Choi (1983). 

We need the following simple result about sufficient conditions for uniform conver- 
gence of the inverse of a sequence of invertible matrix- valued functions A„(x), A„ : M — )■ 
M}^\ to the inverse of its limit A(x). Let || ■ || denote the Euclidean vector and matrix 
norm, respectively. 

Lemma 3.2. Suppose {A(x), A„(x) : n > 1}, is a sequence of k-dimensional matrix-valued 
functions such that 

sup ||A„(a;) - A(x)||2 = o(l). 

X 

If 

0<infcri(a;) and supo"A:(x) < oo, (3.1) 

^ X 

where (Ti{x) (ak{x)) denotes the smallest (largest) eigenvalue of A{x)*A{x), then 

sup ||A^^(x) - A"^(x)||2 = o(l). 
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Theorem 3.1. Fix n G (0, 1). Assume (E). Then, under the null hypothesis Hq, 

Et^£, in D[k,1], 
as T ^ oo, where the a.s. continuous process £ is given by 

S{s) = T] !^B{s) - s-^l'H-^ (1, r/s, . . . , {r/sY)'B{r) drj , 

for s G [k, 1] . 

This theorem provides an explicit formula for the limit process of Et- The limit 
process is a linear function of Brownian motion B[s) and the limit process appearing in 
Lemma 13.11 

3.2 Sequentially updated Residual Process 

Again, \Ts\ denotes the current time point and [TrJ stands for another time point, in 
most cases a previous one. Let us now consider the two-parameter stochastic process 

E\Tr\{[Ts\) =T-^'\Tr\{[Ts\), rG[«:,s], sG[«:,1], 

where for p + I < k < t < T we denote by efc(t) the A;-th residual associated to the 
observation Y^, calculated using the data Yi, . . . ,Yt. This means, having observed the nth 
observation, all residuals are updated. We call E^^r] ( [Ts\ ) the sequentially updated residual 
process. Extend the definition by putting E\Tr\{[Ts\) = if r > s or r, s G [0,k) to obtain 
a -D([0, l]^)-valued process. Note that 

with 

/3[rsj = (^[ts\^ITs\)~'^^[ts\^ITs\ (3.2) 
^[Ts] = (ei, • • • , e[rsj)'- 



Theorem 3.2. Fix k G (0, 1) and assume (E). We have under Hq 

EiTri{[Ts\)^Sir,s), 
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m 1]^; M), as T — )■ oo, where the process £ is given by 



with 



£{r, s) — T] i^B{r) — v{r, s)s ^ J v{u, s)B{u) dv}j 
v(r,s) = {l,r/s,...,{r/syy, 



(3.3) 
(3.4) 



for K, < r < s < 1. 



Notice that the hmit process for sequentially updated residuals has a similar structure 
as for residuals without updating, but the vector functions appearing in the definition of 
£{r, s) now depend on both r and s. Again, the influence of the dependence structure of 
the error terms is summarized by the factor rj. 

Example 3.1. Explicit representations of the limit processes are now easy to obtain. Let 
us consider dimensions p = 1 and p = 2, which are of special importance for applications. 



(i) For p — 1 we have H ^ = 

/ B{r)dr, f 
Jo Jo 



Thus, 



4 -6 
-6 12 

rBir) dr 



and 



^J-;B{r)dr-j,j;rB{r)dr 
-IJ^B{r)dr+'^J^rB{r)dr 



S(r, s) = rj |s(r) + (J - ^) B{u) du+{^j^- ^) uB{u) du^ , 
for r,s & 1], < s. 
(a) Ifp — 2, [l,r/s,r^/s^]'S(r) dr is given by 

f 9j^B{r)dr-fJ^rB{r)dr + fJ^r^B{r)dr \ 

-36 /; B{r) dr + ^ /J rB{r) dr - ^ r^B{r) dr 
^ 30 B{r)dr-^ J^rB{r)dr + ^ J^r^B{r)dr j 



We obtain 



S{r, s) 



, , ,'9 36r 30r2 



36 192r 180r2 



B{r) dr 
rBir) dr 



30 180r ISOr^ 
H r- 



S3 s4 



r^B{r) dr >, 



for r, s e [k, 1], r < s. 
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Rerricirk 3.1. Based on these explicit formulas, simulating trajectories of the process S{r, s) 
becomes a feasible task. Using Donsker's theorem one may simulate trajectories of B{r) and 
employ numerical integration to simulate the moment functions j^r^B{r)dr, s e 
/c e N, appearing in the formulae. 



3.3 Weighted Variance Ratio Process 

We are now in a position to examine the process associated to the sequence of control 
statistics {Ut}- For brevity of exposition we present the results for the sequentially updated 
residuals. The required modifications when using the sequential residuals without updating 
are straightforward. 

Define the kernel-weighted variance ratio process 

Here and in the sequel we agree to put 0/0 = 0. denotes the time point where calculations 
start. To ensure both that the residuals can be calculated and the sums appearing in the 
definition of Vt{s) have a reasonable number of summands for all s e [k, 1], we assume 

p+l < gr < \Tk\ . A plausible choice is 

gT — \Tl\ , for some 7 e (0, k). 
Then gx/T ^ ^ > 0. More generally, let 

7= lim gT/Te[0,K]. (3.5) 
The stopping time Rt can now be represented as 

Rt = rinf{s e [k, 1] : Vt{s) < c}. 

We are now in a position to formulate the main result. 
Theorem 3.3. Fix k G (0, 1) and assume (E). Under Hq we have 

s-^j;{J^Sis,t)dtYKiC{r-s))dr 
Ms)^V{s)- j;S^is,r)dr ' ^ ^ 

in the space D[k, 1]. The limit process is continuous w.p. 1 and depends only on K , Q, and 
Brownian motion B, but not on the quantity rj. 

We discuss this theorem at the end of this section in greater detail. 
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3.4 KPSS (Variance Ratio) Residual Control Chart 

The central limit theorem for the stopping time Rt of the KPSS residual control chart 
appears now as a corollary to Theorem 13.31 

Corollary 3.1. For the stopping time Rt we have under the conditions of Theorem \3.3\ 

Rt/T AtZ = inf{s G [k, 1] : V(s) < Cr}, 

as T —7- oo. 

As a consequence, the KPSS residual control chart can be designed to achieve a given 
nominal significance level a G (0,1). Indeed, Corollary 13.11 implies that Po{Rt < 7") — )■ 
PoiT^ < 1), as T — )■ oo. Since 

7^ < 1 ^ inf V(s) < cr, 

se[K,i] 

we select the control limit as 

CR = F'\l-a), 
where F denotes the distribution function of infsg[K ij V(s). 

Remark 3.2. Having in mind practical applications it is worth discussing the following 
issues. 

(i) Vt converges weakly to a stochastic process which does not depend on any nuisance 
parameter. When a kernel K and the parameter ( are selected, the process V is 
known. This means, the asymptotic law ofVr is distribution- free. As a consequence, 
the asymptotic distribution of Rt is also asymptotically distribution- free. 

(a) In practice, one can simulate trajectories from the limit process and calculate for each 
trajectory the time point where the control limit Cr is reached. In this way one can 
simulate the asymptotic distribution of Rt to determine a control limit cr such that 
the resulting asymptotic type I error rate is a. 
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4 ASYMPTOTIC RESULTS FOR A CHANGE-POINT 
MODEL 

The results of the previous section allow to design monitoring procedures and to study the 
behavior of the resulting procedure under the null hypothesis (in-control model) that the 
underlying time series of observations follows a polynomial regression model with random 
walk error terms under the stated regularity assumptions. 

In this section we discuss the asymptotic behavior of the KPSS residual monitoring 
approach under a change-point model, where the first part of the time series behaves as a 
random walk and the second part is stationary. We assume 

^ ^1 E5=o%, t = o,...,LT^j-i, 

* \ ^TUt, t^lTd\,...,T. 

After the change-point q = \T'd\, which is given by the fixed but unknown parameter 
1? e (0, 1), the error terms change and are no longer a random walk, {^t} is a sequence of 
scale constants satisfying the condition 

~ T^, for some ^ > 0. (4.2) 

Wc shall need a further constraint on /3 which will be discussed below. If /3 = 0, the error 
process after the change, i.e., {et : q < t < T} with q = |_Ti?J, is stationary. However, we 
allow for positive values of In this case the error terms form a row- wise stationary array. 
For simplicity of exposition, we omit the dependence of et on T in our notation. 

Our asymptotic results require the following additional assumptions. 

(CI) {ut} is a strictly stationary process with 

P{\ui\ >x) ^ 
lim — — ■ < oo, 

for some 7 > 2 and satisfies the FCLT 

r-V2 ^ Ui^riB{s), T^oo, (4.3) 

i<lTs} 

for some constant < 77 < 00, where again B denotes standard Brownian motion 
starting at 0. 



18 



Steland 



(C2) The parameters a and (3 satisfy the relations < /3 < 1/2 and 7 > Y/h^- 

Note that the condition on the tail probabilities ensures that the E\ut\^ < 00. 

In the sequel, we use the same notation for the quantities defined for the polynomial 
regression model with error terms {e^} satisfying the change-point model above. 

Let us again start with the residual process. We only discuss the FCLT for the process 
of sequentially updated residuals, E\Tr\{,\Ts\), k < r < s < 1, which is defined as before. 



Theorem 4.1. Suppose the change-point model holds. Additionally, assume that (E), 
(CI), and (C2) are satisfied. Then, for any fixed k, G (0, 1), the following assertions hold 
true. 

(i) We have in the space -D([k, 1]; W), 

Jo 

as T —7- 00. 

(a) The sequentially updated LS residual process converges weakly, 

Et^S.s, zn 1]2;M), 
as T 00, where the cadlag process S is given by 

£^{r, s) = 1] |s(s) - s"^v(r, s)H"^ J v(m, s)B{u) du^ l[«;,^)(s), 
for K < r < s < 1. 

The next result shows that under the aforementioned conditions the asymptotic dis- 
tribution of the kernel-weighted variance ratio process is obtained by replacing formally S 
by in the limit process. 



Theorem 4.2. Suppose the change-point model assumption (E), (CI) and (C2) are 

satisfied. Then, for any fixed k G (0, 1), 

s-'i;U^S,{s,t)dtYK{ar-s))dr 
^'^'^ = i;Slis,r)dr ' ^ ^' 

in the space D[k, 1]. The limit process depends only on K , (, and Brownian motion B, and 
the change-point parameter d. 
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Again, the central limit theorem for the KPSS residual control chart under the change- 
point model appears as a corollary. 



Corollary 4.1. Under the assumptions of Theorem 4-2, the stopping time Rt satisfies 



Rt/T An^ = inf{s e [/€, 1] : V^s) > Cr}, 



as T ^ oo. 



5 SIMULATIONS 

We conducted a Monte Carlo study to investigate the properties of the KPSS monitoring 
procedure when applied to residuals. Time series of length T = 500 according to model 

Yt = /3o + (3i-t+{(3i + A)tl|,,,+i,...}(t) + et, 

where 



T.LiVi, for t<q, 

T.tZlVi + Vt, ioTq<t<T, 



with 



r7i = pr7i_i +6-/36-1, 6'~'iV(0,l), 



were simulated. Let us first discuss the construction of the innovation terms rjt. The AR 
parameter was chosen as p = 0.3 and the MA parameter /3 from the set {—0.8,0,0.8}. 
Thus, {rjt} is a correlated but weakly dependent sequence with mean 0. For time points 
t < q the obervations Yt are given by a random walk with correlated increments r/j. At the 
change-point q the process changes its behavior. The random walk stops and correlated 
error terms T]t determine the behavior of e^. 

Concerning the design of the monitoring procedures we used the Gaussian kernel, 
K{z) = (27r)-^exp(-zV2), z G M, and the bandwidths h e {25,50}, yielding C G {20, 10}. 
The deterministic component of the model is given by a linear trend whose slope, depending 
on A, may change at the change-point, too. If A 7^ 0, there is both a change in the 
error terms and a change in the slope. That should make the detection of the change 
to stationarity of the errors more difficult, since the residuals are estimated assuming a 
constant slope. 
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1000 



Control Limit 

Figure 1. Empirical rejection rates as a function of 10^ times the control limit c for 

^ = -0.8, /3 = 0, and ^ = 0.8 (dashed). 



In a first step, we examined for the setting A = 0, /i = 25, and T — 500, the 
relationship between the control limit c and, firstly, the probability that the method gives 
a signal (Figure 1) and, secondly, the conditional average run length (CARL) given that 
the method gives a signal at all (Figure 2). Since monitoring stops latest at the 500th 
observation, trajectories crossing the control limit later are not taken into account. The 
CARL is the average run length corresponding to all trajectories yielding a signal until 
time 500. The curves, which can also be used to choose the control limit, are quite similar 
for (5 G {—0.8, 0}, but there is an effect for positive values of /3. For the considered setting 
it also becomes apparent that common type I error rates correspond to rather large CARL 
values. On the other side, if the procedure is designed to yield CARL values of, say, 300, 
the chart works on a type 1 error rate which is usually regarded as unacceptable from a 
hypothesis testing viewpoint. However, note that this is partly due to the fact that we 
studied monitoring with a time horizon. Without a time horizon the average run lengths 
would be substantially higher yielding smaller control limits and, as a consequence, smaller 
associated type I error rates. 

We also simulated the power of the KPSS variance ratio residual control chart when 
designed to achieve a type I error rate of a = 0.05. The corresponding control limit was 
obtained by simulating from the limit distributions. We examine the cases A = and 
A = 0.5, where the latter case corresponds to a change to stationary errors term with an 
additional change of the slope. 

Table 1 provides the simulated rejection rates. It can be seen that the KPSS control 
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200 400 600 800 1000 

Control Limit 

Figure 2. Conditional average run length (CARL) as a function of 10^ times the control 
limit c for ^ = -0.8, ^ = 0, and ^ = 0.8 (dashed). 



chart is quite robust with respect to the parameter (5 determining the degree of correlation 
for the increments of the random walk. That behavior is consistent with the theoretical 
and empirical findings in Steland (2007a), where related monitoring procedures for time 
series without trends have been studied in detail. The results show that an early change 
can be detected quite well, but late changes are very hard to detect. However, this is, of 
course, a problem for all methods, and for the statistical problem at hand, N = 500 is not 
a large maximal number of observations. The results also indicate that the power is quite 
robust with respect to additional changes in slope. 



We may summarize that the KPSS control chart for residuals provides a quite reliable 
tool to detect stationary errors in polynomial regression models. 



Table 1. Empirical rejection rates of the KPSS control chart. The right columns 
(A = 0.25) correspond to a change of the slope 
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APPENDIX A: PROOFS OF RESULTS FROM SEC- 
TION 3 

A.l. Proof of Lemma 13.11 

Note that for each s G [k, 1] we have 

/ [Ts] [Ts] [Ts] 

^[ts\^ITs\ = ^ tet, • • • , ^ t^et 
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yielding 



lTs\ 



t=l 



i = l,...,p+l 



T-'/'( [\lTr\/lTs\y-\rridr) 

\Jo / j=l,...,p+l 

{[Tr\/[Ts\y-'T-'/\rridr) 

/ i=l,...,p+l 



(A.l) 



It is straightforward to check that 



Hence, 



sup max|(LTrJ/Lr.jr-(r/.r|=0(l/r). (A.2) 



sup max / {[Tr\/[Ts\y-'z{r)dr - / {r /sy-^z{r) dr\ = 0{1/T). 



0<r<s<l 1<«<P 



JO 



If we define the functional r : {D[k, l],d) — )■ 1], dp) by 

r{z){s)= ( [ ir/sy-h{r)dr) , s €[«:,!], 

V-^o / j=i,...,p+i 

for any z G /^[/t, 1], we obtain 

T-'/'W^Tsi^Tsi^iTsi = r{T-^/h\T.\){s) + op(l), (A.3) 

the op(l) being uniform in s G [k, 1]. It is easy to see that for any sequence {z,Zn} C 
D]^p[k,, 1] such that lim^^oo dp{zn, z) = 0, as n ^ oo, with z G C[k, 1], we have 

lim dp{T{Zn),T{z)) = 0. 

Thus, the continuous mapping theorem in general separable metric spaces (Shorack and 
Wellner (1986), Th. 4, p. 47, and Remark 2, p. 49) and (E) yield the result. □ 

A.2. Proof of Lemma 13.21 

Let cond2(A(x)) = (Tfc(x)/cri(x) denote the condition of A(x) w.r.t. the spectral vector 
norm || ■ ||2. Let £ > 0. If ||A„(a;) — A(a;)||2 < e, the a-priori error estimate for linear 
equations with disturbed coefficient matrices yields 

II -1/ \ -111 cond2(A(a;)) .. . 

^ ^ A(a;) 2 - £^cond2(A(x)) 
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where a^j{x) {a^ ^{x)) denotes the jth column of A^^{x) {A ^{x)). □ 

A.3. Proof of Theorem IHTTl 

Recall the representations 

^lTs\ - P = 0^\Ts\^\Ts\Y''^\Ts\^\Ts\ and e\Ts\ = e[Tsj - ^[ts]01Ts\ - 
where /^ltsJ is defined in (13 ■2p yielding 

= e^Tsi - TV2x|^^jWLr.j(WLr.jT-iX'L^,jXLT.jWLT.j)-^r-3/2WLT.jX'L^,j6LT.j. 
Since x|^^jWltsJ = (1, [Ts\,. . . , [Ts\p)Wits\ = 1' where 1 = (1, . . . , 1)' G W+\ we have 

where 

H^Tsj = Wlt.jT-IxI^.jXlt.jWltsJ. (A.4) 
We will show that the matrix H|^tsJ, which equals 

/ i,jG{l,...,p} 

converges to sH, uniformly in s G [n, 1]. For i = j = 1 this is obvious. Otherwise, i+j > 3, 
and for the corresponding elements of ii[Ts] the assertion follows from 

t=i ^ + J 

Hence 

lTs\ 

yielding 

sup \(U^^rsih-s/it + j-l)\ = Oi[TK\-') (A.5) 
se[K,i] 

for i,j G {1, . . . Recall the representation flA.ll) and flA.3p . Since sup^^j^ ^ T{T~^/'^e^T-] ] 

converges weakly to the random variable 



sup T{r]B){s) = sup (i] / (r/sY ^B{r)dr 
s(i[K.,i] se[K,i]\ Jo /i=i,...,p+i 
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we may conclude that 

sup T-3/^W[^,jX|^,jeLTsj = Op(l). (A.6) 

se[K,i] 

(lA.Sp . (lA.6p . and Lemma [3.21 imply 

Et{s) = T-^'h\Ts\ - l's-iH-ir-3/2wLTsjX|y,jeLTsj +Op(l/LTsJ) 
Using the result flA.Sp we obtain 



Et{s) = T-'/h^Tsi - l's-^H-\{T-^l^e\T.\) + op(l) 

which shows that up to terms of order op(l) the process is a continuous functional of 
{T~^^'^e\Ts\ ■ s E [k,1]}. Consequently, 

Et{s) 4 ri !^B{s) - s-H'H-^ ^ (1, r/s, . . . , {r/sf)'B{r) rfrj , 

in -D[/t, 1], as T — )■ oo. □ 
A.4. Proof of Theorem [Ml 



The proof is similar as the proof of Theorem I3.1[ We have 

e\Tr\{[Ts\) = eiTr] -X.[j.^.^WlTs\iy^[Ts\^[Ts\^[Ts\y^[Ts\y^y^[Ts]^[Ts\^[Ts\- 

Note that 

x'lt.jWlt.j = (1, [Tr\/[Ts\,. . . ,{[Tr\/[Ts\ry 
and let v(r, s) = (1, r/s, . . . , {r / sY)' . Due to flA.2l) we have 

sup ||x|^,jWltsJ -v(r,s)|| =0(l/r). 

0<r<s<l 

Combining this fact with flA.Sp yields 

= "-^^^^^ - {v(r, .).-^H-^ + op(l)} {r{T-'/\Toi){s) + Op(l)} 
where the Op(l) terms are uniform in r, s G [k, 1]. Hence, uniformly in r, s G [k, 1], 

T-i/2%.j(L2^^J) = viT-'/\T.i)ir,s) + opil), 
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where the functional : 1] — t- /^([/t, 1]^; R) is given by 

ip{z){r, s) = z{r) - s~'^v{r, s)U~'^ [ {l,u/ s, . . . ,{u/ sf)' z{u) du, r,sG[K, 1], (A.7) 

for z G D[k, 1]. It is easy to see that for any sequence {z, Zn} C -D[/«, 1] with (i(z„, z) — )■ 0, as 
n — 7- oo, and z G C[fi;, 1], we have ||v5(-2„) — v5(-2)||oo — ?■ 0, as n — )■ oo. Hence, an apphcation 
of the continuous mapping theorem yields 

T-^'\Tr\{[Ts\) 4 ^{aB) = T] |i?(r) - s~\ir, s)U-' J\l, u/s, . . . , {u/sY)'B{u) dv^ , 

as T — )■ cxD. □ 
A.5. Proof of Theorem ISTSl 

We formulate the proof such that the corresponding result for the change-point model 
of Section 4 can be obtained by straightforward modifications. To simplify exposition we 
assume 7 = 0. Note that for any Ai, A2 G M the process 

, \Ts\ i [Ts\ 

W^A„A. (.) = — ^ 5^ (5^ ( [TsJ )) ( (z - [TsJ ) //.) + — ^ 5^ ^ ( [T.J ) 
^\ i=i j=i L-f -^J i=i 

can be written as 

rx.MiE) = Ai(^) l^^l^l^ E^T.i{[Ts\)dz^ Ki{[Tr\ - [Ts\)/h)dr 

where tx^^x^ maps elements of D([0,1]^) to elements of D[0, 1]. Let us show continuity 
of tai,a2 "w.r.t. the supnorm. W.l.o.g. we may assume H-ft'lloo = 1- Using the inequality 

— < + \b\)\ci — b\ for real numbers a,b we can bound |ta^,a2(-2i) — tai,A2(-22)| by 

[\,{T/[Ts\Y\\z^\\^ + \2{T/[Ts\Y{\\z^\\^ + IkslU)] lki - ^2||oo = Oi\\z^ - z^lU- 

Hence, for < si < ■ ■ ■ < < 1, L G N, any associated linear combination X]fc=i PkWx^,x2isk] 
pi, . . . , G M, of the coordinates of the random vectors (Wai,A2('5i), • • • , W^Ai,A2('S-l))) con- 
verges in distribution to ^^=1 PkT\i,\2{^){sk), since Eq- ^ S, T ^ 00, by Theorem 13. 2[ 
This verifies convergence of the finite-dimensional distributions of the 1])^- valued 

stochastic process {Zti,Zt2), where 

Zt,{s) = [Tsr'J2(J2'^(\Ts\)\ m-[Ts\)/h), Zr,{s) = [TsJ ^ ?,( [T.J ) 
i=i \j=i / i=i 
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for s G [k, 1]. Tightness w.r.t. the product topology is a consequence of Prohorov's theorem, 
since both coordinate processes converge weakly. Thus, {Zti, Zt2) — > (^i, Z2), T — 00, in 
{D[k, 1])^, where 

Zi{s) = s-'^ (/ S{z,s)dzYK{C{s-r))dr, Z^is) = 3-^ £\r,s)dr, 
Jo Jo Jo 

for s G [k, 1]. Now a straightforward argument implies that the ratio, Vr, converges weakly, 
as T — 7- 00. Finally, by Lipschitz continuity of the kernel K the limit process V is continuous 
w.p. 1. □ 

A.6. Proof of Corollary [XT] 

The result is shown using similar arguments as in Steland (2007b, Corollary 2.1.) □ 



APPENDIX B: PROOFS OF RESULTS FROM SEC- 
TION 4 

By virtue of the method of proof used in the previous section, we are in a position to extend 
the results for the kernel weighted variance ratio process and its associated stopping time 
to the change-point model of Section 3, if we have a FCLT for the process of sequentially 
updated residuals. Thus we provide a detailed proof of Theorem 14.11 and indicate the 
required modifications to prove Theorem 14. 2[ 

B.l. Proof of Theorem 14.11 

Under the change-point model we have 

{T-i/^e^Tsj : K < s < ^9} 4 {r]B{s) : k < s < ^9}, 
as T — 7- 00. Consider the process T~^^'^eiTs\ for ^ < s < 1. First note that 

T-^/'e^Tsj < sup \T^\tz\\. 
ze[K,i] 

Let S > 0. By assumptions (CI) and (C2) 

p( sup |r-i/2 I ^A^pf j^g^x u I > ) 

\zmi] J \t=im,-,T J 

< (T - [T^\ + 1)P(|mi| > T'^''5/^t) 
= 0(T^~'^(^/2~^)) 

= op(l). 
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if /3 < 1/2 and 7>(l/2 — /3) ^. Again using the Skorohod-Dudley-Wichura representation 
theorem we may assume that 

sup \T-^'h\Ts\-r]B{s)\''AQ, 

Sg[K,1?] 



and 



sup |T / e^Tsjl 0, 



as T — )■ oo. This imphes a.s. convergence in the Skorohod metric to the cadlag process 
-S1[k,i9), i-e., 

as T — )■ oo, which in turn yields weak convergence, 



in -D[/t, 1], as T — i- oo. Combining this fact with (lA.Sp . the continuity of the functional r 
(Jacod and Shiryaev (2003), VI, Proposition 1.22, p. 329) yields 

Jo 

as T — !■ oo. The same arguments as in the proof of Theorem 13.21 show that 
EiTri{[Ts\) = T-'/%Tri{[Ts\) = ^(T-^/^e L^.j ) (r, s) + Op (1) , 
as T — )■ oo, where the functional ip is defined in (]A.7p . We have by linearity 



T-^/^eLTsj -s-V(r,s)H-i f\{u,s)T-^'\\Tu\du 

Jo 

r]B{s) — s~^\{r, s)Yi~^ I \{u, s)r]B{u) du 

Jo 



1[k,i?)(-S) 



+ i?ri(s) -i?T2(r,s) 
= ip{riBl^^^i))){r, s) + Ri{s) + i?2(r, s), 

where the remainder terms are given by 

Rti{s) = T-^'^e\Ts\ - 77S(s)1k^)(s), 

RT2{r,s) = s"V(r, s)H~^ / v{u, s)[T''^^^eiTu] - riB{u)]l[^,^^){s) du. 

Jo 
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Clearly, supsgj^,^) \Rti{s)\ — )■ 0, as T — )■ oo, a.s. To estimate Rt2, denote the maximum 
vector norm and the induced matrix norm by || o \\^ and observe that 

Jo oo 

< / ||v||oo sup \T~'^^^eiTzi - 'nB{z)\l[^^^){s) du 

Jo zGfK.i?) 



< ||v|U^ sup \T-'/hiTzi - vBiz)\ 
ze[K,i?) 

where ||v||oo = sup^^^gf^^^) ||v(r, s) ||oo < oo. Hence, 

sup |i?T2('", s)| < K~"^||v||^||H~"^||oo'*9 sup \T~^^'^eiTz\ — vB{z)\du'^ 0, 
as T — )■ OO. Consequently, 

y.(T-i/2eLr.j)(r,s)'4Xr/m[.,^)), 
as T — )■ oo, which implies via 

d{ip{eiT-\),v{vBl[^,i}))) 0, 

as T — 7- oo, weak convergence which completes the proof. 
B.2. Proof of Theorem [4721 



□ 



To proof goes along the lines of the proof of Theorem 13.31 Notice that now the linear 
combinations J2k=i Pk^^^iM^^k) converge weakly in distribution to J2k=i Pk'^>^lMi^^)i^k)■ 
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